W-20917380 B2C MCP Telemetry by allison-grady · Pull Request #68 · SalesforceCommerceCloud/b2c-developer-tooling

allison-grady · 2026-01-23T01:38:07Z

Summary

Sends telemetry data to verify that tool usage is correctly captured. Modifies the B2C MCP package to send telemetry data to Azure App Insights. Modifies the B2C Tooling package with a reusable telemetry tool.

SDK Operations (b2c-tooling-sdk)

Telemetry Operations:

createTelemetry() - Creates a telemetry instance to allow sending telemetry data logs to Microsoft Azure

Key Features

Collects anonymous usage data (server lifecycle, tool calls, command metrics, environment info) to improve the MCP server and CLI
Sends events to Azure Application Insights: COMMAND_START, COMMAND_SUCCESS, TOOL_CALLED, SERVER_STOPPED, plus exception tracking
Auto-disabled in development: bin/dev.js sets SFCC_DISABLE_TELEMETRY=true which skips telemetry initialization
Production opt-out: Set SF_DISABLE_TELEMETRY=true or SFCC_DISABLE_TELEMETRY=true to disable
Testing override: Set SFCC_APP_INSIGHTS_KEY to use a different App Insights instance (requires SFCC_DISABLE_TELEMETRY=false in dev mode)
Implementation: Connection string in package.json (oclif.telemetry.connectionString), auto-initialized by BaseCommand from SDK, TOOL_CALLED events in src/server.ts, SERVER_STOPPED in src/commands/mcp.ts

Technical Implementation

@salesforce/b2c-dx-mcp: Refactored telemetry to use BaseCommand auto-initialization from package.json, replaced NODE_ENV gating with SFCC_DISABLE_TELEMETRY=true in bin/dev.js, removed src/config.ts, updated README with new env vars (SF_DISABLE_TELEMETRY, SFCC_DISABLE_TELEMETRY, SFCC_APP_INSIGHTS_KEY), and updated tests
@salesforce/b2c-tooling-sdk: Added Telemetry.isDisabled(), Telemetry.getConnectionString(), sendException(), and dataDir option; updated BaseCommand with auto-telemetry initialization from pjson.oclif.telemetry, exception tracking in catch(), and COMMAND_SUCCESS events in finally()

Documentation

Added README documentation in b2c-dx-mcp

Testing

Unit Tests

test/commands/mcp.test.ts: Verifies no --no-telemetry CLI flag exists (telemetry controlled via env var only)
Verifies telemetry initialization passes telemetry instance to server
Verifies server starts successfully without telemetry when disabled
Verifies toolsets are added to telemetry attributes
Verifies Telemetry.isDisabled() respects SF_DISABLE_TELEMETRY and SFCC_DISABLE_TELEMETRY
Verifies Telemetry.getConnectionString() respects disable flags and SFCC_APP_INSIGHTS_KEY override
Verifies telemetry lifecycle events (SERVER_STOPPED, sendException)
test/server.test.ts: Verifies server accepts telemetry in options, works without telemetry, and sends TOOL_CALLED events correctly (success, error result, thrown exception, timing accuracy, disabled telemetry)

Manual Testing

Verified telemetry events by running with explicit App Insights connection string:

SFCC_DISABLE_TELEMETRY=false SFCC_APP_INSIGHTS_KEY="{InstrumentationKey=...}" node ./packages/b2c-dx-mcp/bin/dev.js --toolsets SCAPI --allow-non-ga-tools

then starting and stopping b2c-dx via the Cursor Settings. confirmed events appeared in Azure Application Insights portal

Tests pass (pnpm test)
Code is formatted (pnpm run format)

yhsieh1 · 2026-01-23T17:01:24Z

We should move telemetry to under b2c-tooling-sdk so that it can be shared in other packages. telemetry.ts will need to be refactored to move to SDK. Such as:

Make it configurable - Remove hardcoded PROJECT, make paths/config injectable
Split files - types.ts, telemetry.ts, index.ts
Add factory function - createTelemetry(options: TelemetryOptions): Telemetry
Add dependency - Add @salesforce/telemetry to SDK's package.json
Add barrel exports - Export via SDK's package.json exports

We also need to think about how to provide the config.json that the caller b2c-dx-mcp need to provide. We probably can generated it during CI/CD under packages/b2c-dx-mcp/config.json

yhsieh1 · 2026-01-27T15:26:38Z

@allison-grady Please update the PR description to have detailed information and test steps to give review more context and easy to understand the PR. You could reference other PRs such as #63

clavery

Hi @allison-grady I've got some ideas on how we should support this across all packages in this project. I worked this out as a claude plan and given the size I put it in this this gist: https://gist.github.com/clavery/5afe869ccfbfd9d8df34958f7ded07d1

can you @yhsieh1 review that? Happy to talk through it as well. I incorporated some of the smaller comments I made here into it.

Basically I want to pull this back as far as possible:

Support initialize using our shared BaseCommand
Leverage oclifs configuration in package.json for package specifics (connection string, etc). This also gives a convenient location if we do want to replace in CI/CD or other places
Use oclif CLI dataDir function instead of a non-standard directory
track command life cycle and exceptions in the spot where we are already doing that.

yhsieh1 · 2026-01-30T15:41:01Z

Hi @allison-grady I've got some ideas on how we should support this across all packages in this project. I worked this out as a claude plan and given the size I put it in this this gist: https://gist.github.com/clavery/5afe869ccfbfd9d8df34958f7ded07d1

can you @yhsieh1 review that? Happy to talk through it as well. I incorporated some of the smaller comments I made here into it.

Basically I want to pull this back as far as possible:

Support initialize using our shared BaseCommand

Leverage oclifs configuration in package.json for package specifics (connection string, etc). This also gives a convenient location if we do want to replace in CI/CD or other places

Use oclif CLI dataDir function instead of a non-standard directory

track command life cycle and exceptions in the spot where we are already doing that.

@clavery @allison-grady The gist looks good but I wonder how to prevent local development pollute the production telemetry data? One approach is to disable it by default on local development. We can set SFCC_DISABLE_TELEMETRY=true either in package.json script or override it in dev.js.

clavery · 2026-01-30T19:10:37Z

We can set SFCC_DISABLE_TELEMETRY=true either in package.json script or override it in dev.js.

Just set it in dev.js like Allison had (but use our env vars not NODE_ENV; that was my only complaint about that)

You can set it in tests/e2e tests as well if it happens to trigger there

allison-grady · 2026-01-30T19:17:20Z

We can set SFCC_DISABLE_TELEMETRY=true either in package.json script or override it in dev.js.

To clarify, this will look like:
process.env.SFCC_DISABLE_TELEMETRY = process.env.SFCC_DISABLE_TELEMETRY || 'true';

yhsieh1

It looks good. Thank you!

yhsieh1 reviewed Jan 24, 2026

View reviewed changes

Comment thread packages/b2c-dx-mcp/src/commands/mcp.ts Outdated

Comment thread packages/b2c-dx-mcp/src/utils/index.ts Outdated

allison-grady force-pushed the t/commerce/W-20591276/server-telemetry branch from ef7a38b to 873e507 Compare January 26, 2026 17:16

allison-grady marked this pull request as ready for review January 26, 2026 18:35

allison-grady changed the title ~~W-20917380 B2C MCP Telemetry - Draft~~ W-20917380 B2C MCP Telemetry Jan 26, 2026

yhsieh1 reviewed Jan 26, 2026

View reviewed changes

Comment thread packages/b2c-dx-mcp/src/config.ts Outdated

allison-grady force-pushed the t/commerce/W-20591276/server-telemetry branch from 70f5f27 to 79a8874 Compare January 27, 2026 19:23

yhsieh1 reviewed Jan 27, 2026

View reviewed changes

Comment thread packages/b2c-dx-mcp/bin/dev.js Outdated

Comment thread packages/b2c-dx-mcp/.gitignore Outdated

Comment thread packages/b2c-dx-mcp/README.md Outdated

Comment thread packages/b2c-dx-mcp/README.md Outdated

yhsieh1 reviewed Jan 27, 2026

View reviewed changes

Comment thread packages/b2c-dx-mcp/README.md Outdated

allison-grady requested a review from clavery January 27, 2026 20:22

yhsieh1 reviewed Jan 28, 2026

View reviewed changes

Comment thread packages/b2c-dx-mcp/test/commands/mcp.test.ts Outdated

yhsieh1 reviewed Jan 28, 2026

View reviewed changes

Comment thread packages/b2c-dx-mcp/src/utils/index.ts Outdated

yhsieh1 reviewed Jan 28, 2026

View reviewed changes

Comment thread packages/b2c-tooling-sdk/src/telemetry/telemetry.ts

allison-grady force-pushed the t/commerce/W-20591276/server-telemetry branch from 67df815 to 895791b Compare January 29, 2026 17:23

yhsieh1 previously approved these changes Jan 29, 2026

View reviewed changes

allison-grady dismissed yhsieh1’s stale review via 99cb93d January 29, 2026 19:20

yhsieh1 reviewed Jan 29, 2026

View reviewed changes

Comment thread packages/b2c-dx-mcp/src/config.ts Outdated

clavery requested changes Jan 29, 2026

View reviewed changes

Comment thread packages/b2c-dx-mcp/bin/dev.js Outdated

Comment thread packages/b2c-tooling-sdk/src/telemetry/telemetry.ts Outdated

Comment thread packages/b2c-dx-mcp/src/commands/mcp.ts Outdated

yhsieh1 reviewed Jan 30, 2026

View reviewed changes

Comment thread packages/b2c-cli/package.json

yhsieh1 reviewed Jan 30, 2026

View reviewed changes

Comment thread packages/b2c-dx-mcp/bin/dev.cmd Outdated

clavery previously approved these changes Jan 30, 2026

View reviewed changes

Refactoring changes plus updated tests

594ad50

allison-grady dismissed clavery’s stale review via 594ad50 January 30, 2026 23:06

allison-grady force-pushed the t/commerce/W-20591276/server-telemetry branch from de559f6 to 594ad50 Compare January 30, 2026 23:06

clavery approved these changes Jan 30, 2026

View reviewed changes

yhsieh1 approved these changes Feb 2, 2026

View reviewed changes

allison-grady merged commit 55397bf into main Feb 2, 2026
3 checks passed

clavery deleted the t/commerce/W-20591276/server-telemetry branch February 8, 2026 20:04

Conversation

allison-grady commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

SDK Operations (b2c-tooling-sdk)

Key Features

Technical Implementation

Documentation

Testing

Uh oh!

yhsieh1 commented Jan 23, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yhsieh1 commented Jan 27, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

clavery left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yhsieh1 commented Jan 30, 2026

Uh oh!

clavery commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

allison-grady commented Jan 30, 2026

Uh oh!

Uh oh!

Uh oh!

yhsieh1 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

allison-grady commented Jan 23, 2026 •

edited

Loading

clavery left a comment •

edited

Loading

clavery commented Jan 30, 2026 •

edited

Loading